Solid-state image capture devices such as electronic still cameras and the like rely on an array of discrete light sensitive elements or photosites known as pixels to spatially sample an image of the object and produce sampled output. Because the image is sampled in a discrete manner, images re-created contain false signal content not present in the original object. This phenomenon is known as aliasing and becomes noticeable for those spatial frequency components of the image which are equal to half the spatial sampling frequency along a particular direction of the pixel array, the so-called Nyquist frequency. Although aliasing begins to appear at Nyquist frequency, it is most pronounced at spatial frequencies that are close to the sampling frequency along the given direction of the solid state imager. The effects of aliasing in a monochrome imager (i.e., an imager with no color filter array) with discrete sampling can be seen by reference to FIG. 1 and FIG. 2. The test target shown in FIG. 1 is characterized by a series of concentric rings such that the spatial frequency increases linearly with radius from the center of the target. FIG. 2A is a schematic representation of an array of discrete pixels of a typical solid-state imager. The array has a pixel pitch T along the directions of the x- and y-axes and T/.sqroot.2 along the diagonal directions. If this target is imaged on the discrete array of pixels in the imager, the aliasing pattern shown schematically in FIG. 2B is obtained. (Note that for the sake of clarity, the actual target image has been subtracted and the resultant schematic representation of the remaining aliasing artifact pattern is as shown in FIG. 2B.) It can be seen that, although the aliasing artifacts become evident at spatial frequencies at and above the imager Nyguist frequency (F.sub.N =1/(2T) along the x- and y-axes and .sqroot.2/(2T)) along the diagonal directions), the aliasing pattern is most pronounced in regions of the target image where the spatial frequency is equal to the sampling frequency (F.sub.S =1/T along the x- and y-axes and .sqroot.2/T) along the diagonal directions). In order to reduce or eliminate aliasing artifacts, then, it would be desirable to employ a broad spectral bandwidth optical low-pass, or blur, filter in the optical system which reduces or eliminates those spatial frequencies in the image plane above some cut-off frequency which lies between the Nyquist frequency and the sampling frequency along the direction of the imager for which the sampling frequency is lowest. It should be appreciated that in the case of a monochrome imager little or no information would be lost in a digital imaging system with such a filter in place since it is not possible theoretically to capture information contained in spatial frequencies above Nyquist frequency.
The situation becomes more complicated in the case of color electronic capture devices in which light incident on individual pixels is filtered by one of three primary color filters (i.e., red, green, and blue filters, for example). In many of these color image sensors the spatial sampling frequency for one color is different from the spatial sampling frequency for another; consequently, each color has associated with it a different Nyquist frequency along a given direction. FIGS. 3A and 3B illustrate the effects of aliasing for a typical color solid-state imager with discrete sampling. FIG. 3A is a schematic representation of an array of discrete pixels with green color filters (labeled "G" in FIG. 3A) on half of the pixels and red or blue color filters on the remaining half (labeled "R" or "B", respectively, in FIG. 3A). This particular color filter array pattern is often referred to as the "Bayer" pattern. The pitch of green pixels along the x- or y-axes, T.sub.G, is the same as the pixel pitch T of the sensor while the pitches of the red or blue pixels along the x- or y-axes, T.sub.R and T.sub.B, respectively, are equal to 2T. It should be noted that the pitches of the green, red, and blue pixels along diagonal directions are all equal to .sqroot.2 T. It follows that the green pixel sampling frequency along the x- and y-axes is twice that of the red or blue pixels (F.sub.SG =2F.sub.SR =2F.sub.SB =1/T). FIG. 3B is a schematic diagram indicating the positions of aliasing artifacts only (i.e., captured image minus the image of the test target) for the test target shown in FIG. 1. The most pronounced aliasing artifacts for red and blue pixels occur at spatial frequencies corresponding to the red and blue sampling frequencies along a given direction, whereas all three colors exhibit pronounced aliasing artifacts at spatial frequencies corresponding to the sampling frequency of the sensor. For these imagers it would be desirable to design a broad spectral bandwidth optical blur filter with a cut-off frequency which lies somewhere between the lowest Nyquist frequency (i.e., the red or blue Nyquist frequency along the x- or y-axes for the Bayer-type imager) and the highest sampling frequency (i.e., the sensor sampling frequency along the x- or y-axes). In this case, a trade-off must be made between the degree of aliasing suppression and the degree of image sharpness since information contained in those spatial frequencies of the image between the cut-off frequency of the optical blur filter and the highest Nyquist frequency (i.e., that associated with the green pixels along the x- or y-axes) will be lost.
Phase noise optical low pass filters have been disclosed in the literature for applications as photographic softening filters as well as anti-aliasing filters in digital cameras. See "Optical Noise Filter" by K. Sayanagi, Journal of Applied Physics (Japan), 26, No. 10, 1958, pp 623-632; and "Optical Phase-Noise Filter for Color Portrait Photography" by Y. Okano, Proceedings of the International Commission for Optics, 13th Conference Digest--Sapporo, Japan, 1984, pp 104-105. These articles describe a phase-noise optical low-pass filter formed by a transparent substrate 10 with a plurality of randomly positioned and mutually spaced transparent spots 11 each spot being of a thickness on the order of the wavelength of light passing through the filter as shown in FIG. 4A. FIGS. 4B and 4C show side views of the phase-type optical low-pass filters for cases where the randomly placed transparent spots are formed by projections from the surface of the transparent substrate (FIG. 4B) and alternatively by depressions in the surface of the transparent substrate (FIG. 4C).
When positioned between the objective lens and the image plane of the imaging system (see FIG. 5A) or directly in front of the objective lens (see FIG. 5B), such a phase noise-type low-pass optical filter causes aberrations in the wave front of the light passing through it. The modulation transfer function (MTF) due to the blur filter alone (i.e., excluding the effects of lens aberrations and finite aperture) can be derived from the auto-correlation of the randomly arrayed transparent spots where each spot introduces a phase difference .phi..sub.j relative to light passing through the substrate in regions without spots. The MTF at a particular optical wavelength .lambda..sub.j, as a function of spatial frequency, is given as follows: ##EQU1## where f is the spatial frequency, .xi. is the fractional area of the surface of the filter that is covered by the transparent spots, ##EQU2## and n(.lambda.) is the index of refraction of the material forming the spots at the specific wavelength .lambda., n'(.lambda.) is the index of refraction of the medium surrounding the spots at the specific wavelength .lambda.; and t is the physical thickness of the transparent spots. The function g(.lambda.bf/2) is the geometrical auto-correlation of the spots along a specific direction and has the properties g(0)=1 and g(.lambda.bf/2)=.xi. for very large values of the argument .lambda.bf/2. The function g depends implicitly on the phase spot diameter a in that the auto-correlation of the phase spots drops off more rapidly with spatial frequency when the phase spot diameter is smaller. The quantity b in EQN. (1A) is the distance between the spot-bearing surface of the phase noise filter and the image plane if the filter is placed in "image space" (i.e., the space between the imaging lens and the image plane). It becomes equal to the focal length of the imaging lens if the filter is placed in "object space" (i.e., the space between the imaging lens and the object) or within the imaging lens system itself.
FIG. 6 shows a plot of MTF(f,.lambda.) for a particular wavelength. As can be seen, the function exhibits three distinct regions as the spatial frequency increases from f=0 1p/mm to very high spatial frequencies. In the low spatial frequency region (first region) MTF(f,.lambda.) decreases monotonically with f until it reaches an intermediate spatial frequency region (second region) where the function begins a damped oscillation. The oscillations die out and the function approaches the asymptotic value EQU M.sub.c (.lambda.)={1-[2.xi.][1-cos .phi.(.lambda.)][1-.xi.]}(2)
as the spatial frequency becomes very large (third region), where M.sub.c (.lambda.) is referred to as the MTF cut-off function. Although MTF(f,.lambda.) approaches M.sub.c (.lambda.) as f approaches infinity, there is a smaller finite value off for which MTF(f,.lambda.)=M.sub.c (.lambda.). This value of f marks the transition from the low (monotonically decreasing) to the intermediate (damped oscillatory) spatial frequency regions and is given by the expression ##EQU3## 1 p/mm where F.sub.a (.lambda.) is the canonical cut-off frequency of the filter. It should be appreciated that in order to reduce aliasing artifacts, it is desirable to design the phase noise filter so that the MTF is sufficiently small for spatial frequencies at or above the appropriate sampling frequency as discussed previously. From EQN. 2 it can be seen that M.sub.c (.lambda.)=0 can be obtained providing .xi.=0.5 (i.e., the substrate surface has 50% spot coverage) and the physical thickness of spots is chosen be ##EQU4## which is the physical thickness that causes the phase difference due to the spots to become an odd multiple of .pi. radians, EQU .phi.(.lambda.)=(2m+1).pi.. (4B)
The quantity m in EQNS. 4A and 4B can be 0 or any positive or negative integer. Typically the filter canonical cut-off frequency F.sub.a (.lambda.) is adjusted such that the MTF of the entire optical system (i.e., including lens) is sufficiently low at and above spatial frequencies where aliasing artifacts become problematic while simultaneously maintaining as high a value possible at spatial frequencies below Nyquist.
It is known to construct optical low-pass filters for the purposes of intentional image blurring. See for example, U.S. Pat. No. 4,480,896 which discloses a transparent random spot blur filter for use in portrait photography. In this patent, the value of t in EQN. 4A is chosen for a particular wavelength in the green region of the visible spectrum. This is done in order to cause maximum blurring for the wavelength at which the human is eye is most sensitive, thereby reducing the sharpness of undesirable features in portrait photographs such as facial blemishes. However, this patent suffers a serious disadvantage for anti-aliasing applications in electronic cameras since the green channel is precisely the one that carries most of the luminance information. Consequently it is important to retain as much resolution as possible in the green channel. On the other hand, no provision is made in this patent to blur the red and blue components of discretely sampled images in order to reduce aliasing for these color bands.
U.S. Pat. No. 2,959,105 is similar to the previously mentioned patent in that it discloses the use of a random spot phase noise filter in combination with an ordinary photographic lens to obtain "soft" focus in portrait photography. As a further objective this patent mentions the use of the phase noise filter as a spatial frequency cut-off device to produce continuous tone photographic images from any optical images of periodic structures such as screen dots or of line structures in television images, etc. This patent does not anticipate the adaptation of the phase noise filter to solve problems specifically associated with elimination of aliasing artifacts in modern solid-state electronic cameras with discrete sampling.
Another example from the prior art is U.S. Pat. No. 5,280,388 which describes a wavelength selective phase grating optical low-pass filter for use with electronic still cameras or other discretely sampling image capture devices. More specifically, the filter described in this patent is intended for use in color solid state image capture devices which incorporate color filter array patterns such as the Bayer pattern which sample more in the green portion of the visible spectrum than in the red or blue portions. In this patent, the randomly placed transparent spots are replaced by a regular two dimensional transparent phase grating pattern. Wavelength selectivity is accomplished by appropriately choosing the refractive indexes and the refractive index dispersions of the transparent phase grating material and the transparent material in which the phase grating is embedded such that the refractive indexes are matched for a particular wavelength in the green spectral band and mismatched in the red and blue spectral bands. That is to say n(.lambda..sub.G) and n'(.lambda..sub.G) in EQN. 1B are equal so that, according to EQN. 2, M.sub.c (.lambda..sub.G) is equal to unity where .lambda..sub.G is a particular wavelength in the green spectral band. Furthermore the thickness of the stripes that form the phase grating is chosen so that, given the refractive index dispersion differences between the grating material and the material surrounding the grating, M.sub.c (.lambda.) is sufficiently close to zero for certain wavelengths in the red and blue portions of the visible spectrum. This patent also suffers several disadvantages. First of all, the gratings must be aligned precisely to the array axes of the pixels which comprise imaging area of the solid state image capture device. This invention has the further disadvantage that special conditions are placed on the physical properties (i.e., index of refraction and dispersion) of the materials used to fabricate the low-pass filter. These properties are restrictive and may result in conflicts with other requirements such as the desire for low manufacture costs. The need to form two separate gratings oriented perpendicularly to one another also complicates the fabrication and adds to the expense of fabricating the device.
In addition to the specific problems mentioned in connection with each of the three prior art references already referenced, there is a fundamental problem which none of these patents address for applications involving anti-aliasing in electronic still cameras or other discretely sampled digital image capture devices. The difficulty can be understood most easily by referring to FIG. 7. This figure shows three hypothetical spectral sensitivity functions filters S.sub.R (.lambda.), S.sub.G (.lambda.), and S.sub.B (.lambda.) that characterize the red, green, and blue channels, respectively, of a typical color digital image capture device. These spectral sensitify curves represent the response of each of the individual color channels to incident light as a function of wavelength. They include characteristics such as the spectral dependence of the quantum efficiency of the photo-diodes or photo-capacitors that comprise the individual pixels as well as the transmission characteristics of the color filters associated with the three color channels. For the purposes of the present invention, these spectral sensitivity curves also include the spectrum of the incident illumination. Since each of the spectral sensitivity curves contains wavelengths over a broad spectral band, the proper MTF of the blur filter at cut-off for a particular color filter must be obtained by a weighted average over the spectrum of that color channel. In the case of the phase noise blur filter, EQN. 2 must be spectrally averaged to yield EQU M.sub.cR .intg.=S.sub.R (.lambda.)M.sub.c (.lambda.)d.lambda.(5)
where M.sub.cR is the spectrally averaged MTF cut-off function for the red channel. Similar expressions are obtained for M.sub.cG and M.sub.cB for the green and blue color channels, respectively. According to EQN. 1B, the larger the thickness t of the transparent spots, the more the value of .phi.(.lambda.) will vary over the wavelength band of a given channel. As a direct result of the variation of the phase difference over the bandwidth of the color channels, the values of M.sub.cR, M.sub.cG, and M.sub.cB will be significantly larger than zero. Consequently the phase noise blur filter will not be effective in reducing aliasing artifacts present in the captured image as t becomes larger. Not one of the three prior art patents referenced above solves the problem of aliasing suppression over broad spectra in color solid state digital cameras.
U.S. Pat. No. 4,009,939 relates to a wavelength selective low-pass optical filter that is used to eliminate aliasing artifacts in color television camera systems. This patent does disclose a way for solving the problem mentioned in connection with the previous prior art patents; that is, it addresses the problem of achieving aliasing suppression across broad spectral bandwidths. The filter is comprised of a first transparent phase grating, a second transparent phase grating, and a transparent substrate supporting the first and second phase gratings. Each phase grating is formed by a plurality of parallel transparent stripes or laminae such that the laminae of the respective first and second gratings are non-parallel to each other crossing at an angle between 90.degree. and 160.degree.. The thickness of the laminae in the first phase grating, t.sub.1, and the second phase grating, t.sub.2, are selected independently and in such a way that the first phase grating attenuates high spatial frequencies in the red spectrum and the second phase grating attenuates high spatial frequencies in the blue spectrum. Accordingly, the phase retardation of the laminae in the first grating, .phi..sub.1, and the second grating, .phi..sub.2, are such that the MTF of the optical imaging system which includes such a low-pass optical filter is sufficiently close to zero for the respective design wavelengths. In this case .phi..sub.1 and .phi..sub.2 are defined according to the formulae ##EQU5## where n.sub.1 and n.sub.2 are the indexes of refraction of the laminae; .lambda..sub.R and .lambda..sub.B are the design wavelengths for the red and blue spectral bands, respectively; and n' is the index of refraction of the medium surrounding the stripes. In other words, the thicknesses of the laminae of the first and second gratings are chosen to cause the MTF associated with each grating individually to cut-off high spatial frequency components in the red and blue images, respectively. In this regard, a double layered optical low pass filter can be designed with spectral averages M.sub.cR and M.sub.cB that are both very low by choosing t.sub.1 and t.sub.2 such that .phi..sub.1 and .phi..sub.2 are each individually equal to .pi. for specific wavelengths in the transmission spectra of the red and blue filters, respectively. In this case, all three cut off functions M.sub.cR, M.sub.cB, and M.sub.cG have low values and consequently are effective in suppressing aliasing artifacts.
Unfortunately the invention in U.S. Pat. No. 4,009,939 suffers from several disadvantages. The low-pass filter disclosed in this patent reduces aliasing artifacts only along one direction in the image plane. While this may be of use in color television camera systems, it is inadequate for application in electronic image pick-up devices with discrete sampling which require suppression of aliasing artifacts in two-dimensions. Furthermore, this filter would be difficult to manufacture in that two gratings each with its own precise laminae thickness must be formed and aligned with respect to one another. Finally, the unique axis of this low-pass filter along which the anti-aliasing function is achieved must be aligned properly with respect to an axis of the electronic image capture device (i.e., the x- or y-axis of the array of pixels) which means that more cost is added to the manufacture process.
U.S. Pat. No. 5,585,885 describes the use of an optical low pass filter for use in a photographic camera exposure control system. This low pass filter is comprised of circular phase components of randomly distributed convex and concave surfaces on a transparent substrate such that the height of the phase components represents a phase difference between the light passing through the phase components and the other areas of the filter that is 1/2 its wavelength. This phase difference corresponds to .phi.=.pi. radians and the height h is given by the relationship ##EQU6## where n is the index of the material which comprises the phase components and the phase components are assumed to be surrounded by air. The specific value of the optical wavelength .lambda. which occurs in EQN. 7 is specified in the prior art patent only to the extent that it is characteristic of the measured light passing through the phase noise filter. The object of the invention in U.S. Pat. No. 5,585,885 is to provide a camera photometer that can precisely measure the light distribution of a secondary photo image regardless of the incidence of the light from the photographic subject when the photometric sensors have been divided into multiple units. This optical low pass filter is designed specifically to be used in a camera system whose primary image capture mechanism is not specified but whose secondary image capture system is an array of discrete photometric sensors. The filter is placed optically upstream of a secondary image-forming lens and is used to blur the image formed on the array of photometric sensors. In this way, light that is focused in the non-sensing areas between photometric sensors will be blurred enough so that the brightness of this light can be measured by the photometric sensors adjacent to these non-sensing areas. After conversion of light to electric signals, the signals are used to determine the light exposure for the primary image capture function of the camera.
U.S. Pat. No. 5,585,885 makes no mention of an application involving suppression of aliasing artifacts across broad spectral bandwidths in digital electronic capture devices. The purpose of the prior art invention is to enable accurate measurement of the brightness distribution of the photographic subject and this information is used to set the camera exposure for capture of the primary image. Clearly the purpose of this invention is distinct from that of the present invention. Furthermore, U.S. Pat. No. 5,585,885 specifies randomly distributed circular phase components having concave and convex surfaces and a diameter a defined by the relationship ##EQU7## where f is the focal length of the secondary photometric image-forming system, T is the photometric sensor spacing, F.sub.s is the sampling frequency associated with the photometric sensors, and .lambda. is a wavelength of light that is characteristic of the light passing through the filter. In this case, the diameter of the phase spot is chosen to be very small in order to diffract light to very large angles thereby maximizing the possibility of collecting the light by the array of photometric sensors.